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(54) Abstract Title 

Digital controlling system for electronic lighting devices 

(57) A digital real-time, controlling system for electronic lighting devices that analyses audio information to 
determine key indicators such as timing, texture, feel, timbre and style. The analysed information can be used 
to control electronic lighting devices and to schedule sets of commands for electronic lighting devices based 
on analysis of the audio information. The result is a digital real-time electronic lighting device controller that 
synchionises lighting control to the audio performance sympathetically with a high degree of autonomy, yet 
still provides a varied and context sensitive lighting show. 
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DIGITAL CONTROLLING SYSTEM FOR ELECTRONIC LIGHTING 

DEVICES 

Field 

This invention relates to a digital controlling system for electronic lighting 

5 devices. 

Background 

Electronic lighting systems have developed greatly both in complexity 
and performance over the past few years. Most stage, TV or night club lighting 
systems operate with a number of very versatile, electronically operated lighting 

10 fixtures thai provide exciting visual effects such as panning, tilting, flashing and 
rotating. The entirety of individual fixtures - normally combined with a lighting 
console - comprise a lighting system. Lighting systems are in the main, operated via a 
lighting console which provides commands to each of the fixtures and can be 
progranmied to achieve certain effects at specific times. Very powerful effects can be 

15 achieved via pre-programmed lighting routines, which are normally progranuned to 
complement a musical or sound performance. Lighting Directors and/or console 
operators use faders and other controls on the console to create a show that uses 
Hghting, smoke and other visual effects during the performance to enhance or 
highlight parts of the audio performance. It is important that the lighting effects are in 

20 time wdth the audio performance to maximise the effect and one of the main aspects of 
a well created lighting show, is the manner in which it relates to an audio 
performance. 

A lighting operator may create a lighting show by creating routines, 
saving them on the console and overlaying them until the entire lighting show is 

25 synchronised. These type of pre-programmed lighting routines can provide lighting 
effects that are perfectly synchronised to the audio performance v^en the audio 
performance is provided by tape, record, computer, CD or other forms of absolute 
reproduction. This is because the lighting routines are programmed against an 
accurate time code generated by the console or some external source. In many pre-set 

30 performances, a lighting operator will also use impromptu manual controls on the 
console in conjunction with the pre-programmed routine, to create a more live feel to 
the pre-set lighting show. Pre-programmed routines work well for predetermined 
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time-accurate performances as above, but by nature can not cope well with 
performances where the audio may change style, timing, timbre or feel without any, 
or with little warning. This happens more than is commonly thought, musicians may 
add or leave out musical phrases, or may extend a song. Even in the nightclub 
5 environment where the music is essentially based on the same, repetitive beat, a DJ 
may scratch or spin a record for audio effect that changes the essential beat of the 
performance. Part of a DJ's role is to change records according to how he or she feels 
the audience will react, in this case the timbre or feel of the music is paramount and 
the lighting effects can make or break the effect a DJ is trying to achieve. 

10 Commonly a lighting operator will use a combination of pre-programmed 

routines and manual control of the hghting console to provide a light show that is 
sympathetic to a live audio performance. This may involve setting a general colour 
theme for a particular part of an audio performance with individual fixtures set to 
flash or move in time with the beat, perhaps by setting the fixture to *soimd to light' 

15 mode or by setting an automatic flash on the console which is adjusted to be in time 
with the audio performance. Some consoles offer a 'time bend' facility so that the 
operator can adjust the timing of a pre-set lighting show to keep it in time with the 
audio performance- Different pre-set lighting themes are available fiiom the console 
so that an operator can switch themes in time with the audio performance. Naturally 

20 the amoimt of live effects an operator can add is limited and live effects are prone to 
timing problems. 

Lighting Directors and/or operators are often employed for larger audio 
performances of all musical types, whereas the smaller perfomiance is more likely to 
rely on the individual sound synchronising capabilities of individual lighting fixtures 

25 which are minimal. This is relevant to both temporary or mobile stage performances, 
or in fixed installations such as night-clubs or studios. Even in larger clubs or studios, 
it is more often the case that the entire lighting system is used to only a fraction of its 
capabilities with operators usually changing pre-set themes to change the lighting 
mood vsdth a degree of manual override for extra lighting effects such as strobe or 

30 lasers, 

A lighting show intended to enhance an audio performance needs to be in 
time and in sympathy with the audio whilst offering a myriad of effects that regularly 
change. Performers and lighting operators are constantly striving for new special 
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effects and closer integration of sound and light to create more effective overall 
performances. Current lighting systems do little to help an operator tie the lighting 
performatice to the timing and feel of the audio, preferring to base events on a fixed 
time basis. 

5 Further to this, current lighting systems do not have the ability to examine 

an audio performance to enable it to generate a light show that is linked to the timing, 
feel, and texture of an audio performance and create a sympathetic lighting show. 

In addition, lighting consoles are dedicated pieces of equipment, normally 
expensive and cxmibersome, for the smaller artists or lighting operators it is 
10 impracticable to include a lighting system in a music making set-up or studio, 
meaning that they can not arrange or produce the light show as they may do with 
music. 



SUMMARY OF THE INVENTION 
A first aspect of the invention provides a digital real-time, controlling 
15 system for electronic lighting devices comprising a computer 70, one or more 
information input devices 3 to receive and convert audio information fi-om audio 
sources 1, one or more controlling output devices 60 to control electronic lighting 
devices 62, and a software program 80. 

The software program 80 analyses the audio information from the 
20 information input devices 3 to determine key indicators including but not limited to 
the timing, timbre, instrument identification, style, amplitude, fi^equency, rate-of- 
change, diiration, tone, occurrence and feel of an audio performance. The resulting 
analytical data is used to create triggers in the form of data bus 10 which can be used 
as a time clock for sequenced events and commands. Additionally the analytical data 
25 can be used to determine the texture or feel of the audio information to create 
additional commands, event triggers or schedules. The analytical data, or derivatives 
of it can also be patched to any controllable feature of any lighting device 62, The 
result is a digital real-time electronic lighting device controller that synchronises 
lighting control to the audio performance with a high degree of autonomy, yet stiii 
30 provides a varied and context sensitive lighting show. 
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Another aspect of this invention is the use of data bus 10 that can be 
treated with logical, mathematical or other functions and used to cc»nmand the system 
to change overall hghting effects in synchronicity with the audio performance. 

Yet another aspect of the invention is the creation and recognition of a 
5 real-time, time code that contains key indicator information relating directly to the 
audio performance. This aspect of the invention provides a means of remotely 
operating equipment via a suitable medium for example the internet or wireless, and 
further, that such time code or data stream may be embedded or included in musical 
or other recordings or saveable data formats 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

These and other more detailed and specific objects and features of the 

present invention are more fully disclosed in the following specification, reference 

being had to the accompanying drawings, in which: 

FIGURE 1 is a block diagram of a general embodiment of the system, 
15 with a more detailed preferred embodiment of the input and output sections of the 
system. 

FIGURE 2 is a detailed block diagram of a preferred embodiment of the 
audio analysing and trigger generation section of the system. 

FIGURE 3 is a detailed block diagram of a preferred embodiment of the 
20 first level of operational control, "performance". 

FIGURE 4 is a detailed block diagram of a preferred embodiment of the 
second level of operational control, "acf. 

FIGURE 5 is a detailed block diagram of a preferred embodiment of the 
third level of operational control, "scene". 

25 FIGURE 6 is a detailed block diagram of a preferred embodiment of the 

fourth level of operational control, "mood". 

FIGURE 6b is a more detailed block diagram of a preferred embodiment 
of one aspect of the fourth level of operational control, "mood". 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Figure 1 shows a preferred embodiment of the system, comprising; 

A computer 70 on which the analysing and controlling software 80 
5 resides; 

One or more information input devices 3, which are electronically 
linked to one or more audio sources 1 via one or more connections 2; 

One or more output devices 60 are electronically linked to electronic 
lighting equipment 62 which may be one or more individual lighting 
10 equipment device via one or more connections 61 ; 

The audio sources 1 can be in analogue or digital electronic form such as 
are generated by microphones or other acoustic transducers, audio mixing or treating 
devices, broadcast, computer generated or as generated by any other sound generator 
devices. Audio sources 1 may be separate sources such as one source for each 
15 instrument or musical element of an audio performance, or composite audio sources 
of an audio performance such as stereo 'left and right', or any combination thereof 

Connections 2 can be electrically conductive wires, fibre optic cables, 
wireless links or any combination thereof Furthermore connections 2 may utilise a 
combination of commimications interfaces such as the Internet or modem 
20 connections. 

Input devices 3 may be computer sound cards, multiple input DSP cards 
or other internal or external audio input devices. Input devices 3 may feature one or 
more audio inputs and such inputs may be in analogue or digital form via wire, fibre 
optic cable, wireless links or any combination thereof The resulting data stream from 
25 input devices 3 forms input data bus 4, 

The following FIGURES illustrate processes undertaken, and preferred 
embodiments of the software system 80, of the present invention. 

As illustrated in FIGURE 2, input data bus 4 is fed to the audio analysing 
section 5 of the software system 80. The audio analysing section 5 of the preferred 
30 embodiment of the present invention is an FFT spectral analyser but can be any form 
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of analyser including but not limited to, sampling, wavelets, or pattern identification 
analysers- Data from the spectral analyser 5 forms spectral data bus 6. 

A plurality of trigger channels 7 receive analysed information from 
spectral data bus 6 whereupon parameters can be changed and functions performed by 
5 function & parameter controls 1. 1 and triggers generated by trigger generator 7,2. An 
important function of trigger channels 7 is to monitor a specified frequency band of 
the audio information from sources 1 and create triggers according to the function & 
parameter control 7.1 settings. The inventor recognises that the extent that the 
function & parameter controls 7.1 monitor and affect the analysed information 6 is of 

10 great benefit to the ability to create clear and relevant triggers via trigger generators 
7.2. Accordingly, the function & parameter controls 7.1 comprise adjustable 
functions, parameters and features including, but not limited to; central frequency to 
monitor, 'Q' or width of the central frequency band, amplitude or level adjustment 
and compression, minimum and maximum threshold, variable noise gate, and other 

15 sound separating or identifying techniques. Furthermore, the function 8c parameter 
controls 7.1 may contain sample and lookup functions whereby certain sets of 
monitored audio information are compared to a library of stored sets for identification 
and eventual trigger generation. Function & parameter controls 7. 1 also provide the 
facility to receive data directly from a specified input or plurality of inputs of input 

20 device 3 thereby by-passing spectral analyser 5, this Direct Injection mode (DI) 
enables one or more specified audio source 1 to be fed directly to one or more 
specified trigger channel 7. Trigger generators 7.2 create a trigger based on the 
information that has been affected and/or treated based on the function and parameter 
setting of the function & parameter controls 7,1. Triggers from the trigger generators 

25 7.2 form data bus 1 0, 

FGURE 3 illustrates a preferred embodiment for the first level of 
operational control titled ''performance" 20, which is the highest level of operation 
and contains its own set of sub levels as described later in this description. A 
performance 20 contains a timing analyser 20.1 and a set of function, parameter & 
30 main controls 20.2. An altemative embodiment also contains a texture detector 31.1. 
Data bus 10 provides performances 20 with triggers from trigger channels 7, which 
are analysed by timing analyser 20. 1 to provide timing information to timing bus 25. 
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Timing analyser 20.1 calculates the main timing elements of the audio 
sources 1 which includes but is not hmited to Beats Per Minute (BPM) or time 
signature by analysing the triggers in data bus 10. An altemative embodiment 
analyses the information in spectral data bus 6 to calculate BPM. Timing analyser 
5 20.1 also creates other timing information including but not limited to; multiples or 
divisions of BPM, BPM count, and pre or post delay timing information which may 
be used to compensate for data transmission delays or delays in response by lighting 
devices 62. Timing analyser 20.1 also includes a beat continuation function which 
uses BPM timing calculations to maintain a BPM simulation, even whilst a section of 
10 audio from audio sources 1 is analysed as containing no BPM information. This 
function enables the software system 80 to continue to provide lighting control that 
maintains synchronicity with the audio sources 1 even whilst some or all of the 
elements of audio source 1 being used to define BPM are not present for a period. 

Furthermore, timing analyser 20. 1 contains a BPM counter reset function 
15 whereby manual and automatic reset options are available to ensure that the BPM 
counting function counts its Most Significant Beat (MSB) in synchronicity with that 
of audio sources 1 . Automatic MSB reset may be instigated by a number of options 
including but not limited to time code reset indicators, control instructions, or the 
process whereby zero BPM information preceding an identified beat is utilised, 

20 Furthermore, timing analyser 20. 1 provides and/or reads a number of time 

codes for remote operation or linked operation of systems, such time codes may 
include; SMPTE or MIDI, or a time code generated by the present invention. 
Furthermore such timecodes, may be sent, received, transmitted, saved or embedded 
in all forms of recording medium. Data from timing analyser 20. 1 forms timing bus 

25 25. 

Function, parameter & main controls 20.2 comprise a set of main, 
parameter and function controls for; overall system operation for example start and 
stop, file management tools such as load and save, and configuration of time code and 
the overall system. In addition to the above, commands for the control bus 26 relating 
30 to manual access or overrides, and communications are generated by function, 
parameter & main controls 20.2. 
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Furthermore, function, parameter & main controls 20.2 provide a function 
whereby a schedule can be generated to instruct the ne?ct level of operation titled "act" 
31. This fimction contains a list of available acts 31, which may be instructed to 
apply, and a time scale - measured in convenient time imits such as minutes or bars - 
5 against which, act change or select instructions can be set. All data from function, 
parameter & main controls 20.2 forms control bus 26. 

FGURE 4 illustrates a preferred embodiment for the second level of 
operational control titled "act" 31. Timing bus 25 and control bus 26 are fed to act 
selector 30 to provide a means of selection of acts 31. Control bus 26 provides act 
10 selection instructions created in performance 20, and timing bus 25 provides timing 
information which is correlated by act selector 30 with instructions from control bus 
26, which can ensure that act selection is effected in time with the BPM or time 
indicator of audio sources 1 . 

Acts 3 1 comprise a plurality of texture detectors 31.1 and one or more 
15 scene arranger 35 which is a means of arranging the next level of operation titled 
"scene" 41. With overall act selection being commanded by act selector 30 as 
previously explained, the acts 3 1 create scene selection instructions in two ways, both 
methods may be based on data supplied by timing bus 25 and data bus 10. 

The first method operates as follows: Scene selectors 35 contains a list of 
20 available scenes 41 and a time scale against which certain scenes 41 may be instructed 
to apply. The period of time over which scenes 41 are selected by scene arranger 35 
is measured in units directly related to the BPM or time signature of audio soxirces 1 
as calculated by timing analyser 20,1 and supplied by timing bus 25, The result is a 
set of scene change instructions created by scene arranger 35 and sent to scene bus 36, 
25 which are in time with the BPM of audio source 1 . 

The second method that acts 31 uses to create scene selection instructions 
is to autonomously and automatically select scenes according to the settings of texture 
detectors 31.1, explained as follows; 

Texture detectors 31.1 comprise function & parameter controls 31.2 and 
30 trigger generator 31.3. Function & parameter controls 31.2 comprise a set of 
definable filters, function, and parameter settings or options that may be used to 
identify certain aspects of audio source 1. A first aspect of function & parameter 
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controls 3 1 .2 is to identify certain elements including but not limited to the following; 
beats, treble, middle or specific audio band activity, silence, crescendos or other 
musical nuances, and specific pattems or instrument characteristics. These elements 
may be identified by analysing the triggers in data bus 10, by analysing the data in 
5 spectral bus 6, or by sample and look-up or other methods, and may be used in inverse 
to find missing elements. A fiarther aspect of fimction & parameter controls 3 1 .2 is to 
allow the specification of a detection time period which directly relates to the BPM of 
audio sources 1 - available fi^om timing bus 25 - in which to identify the 
aforementioned elements. Yet another aspect is pre and post delay and duration 
10 settings for the detection period, said settings again related to the BPM or timing 
signature of audio sources 1 as available from timing bus 25. 

Trigger generator 31,3 additionally provides scene change triggers to 
scene bus 36 based on information supplied to, and having been affected by, the 
settings of function & parameter controls 31.2. The scene change instructions created 
15 by this method may be set to override those created as described in the first method. 

Scene change instructions from both methods described above form scene 

bus 36. 

FIGURE 5 illustrates a preferred embodiment of the third level of 
operation titled "scene*' 41. Control bus 26 and scene bxis 36 are fed to scene selector 
20 40 to provide a means of selection of scenes 41 according to selection data as 
generated by act 31 and command data as created in performance 20. 

Scenes 41 comprise function & parameter controls 41. 1, and mood 
arranger 41.2 which provides a means of arranging the next level of operation titled 
"mood'' 51. Mood arranger 41.2 provides a means of selecting, firom a list of 

25 available options, which moods 51, are to be used for a particular scene 41, and sends 
mood selection instructions to mood bus 45. Function & parameter controls 41.1 
provide a means of defining from a Hst of options, how a scene 41 will use the chosen 
moods. Examples of such options being; use a certain mood 51 every time scene 
41(n) is applied, i.e. cycle, or; use a certain mood 51 only the first time scene 41(n) is 

30 applied, i.e. one-shot. Mood selection instructions from scenes 41, form mood bus 
45. 
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FIGURE 6 illustrates a preferred embodiment of the fourth level of 
operation titled "mood" 5 1 . Control bus 26 and mood bus 45 are fed to mood selector 
50 to provide a means of selection of moods 51 according to selection data as 
generated by scenes 41 and command data as created in performance 20. 

Moods 51 comprise function & parameter controls 51.1 and an output 
arranger 51.2 which provides a means of arranging data to be sent to lighting devices 
62 via output bus 55, output devices 60 and connections 61 . A preferred embodiment 
of the present invention uses a plurality of moods 51, for each individual lighting 
device 62. 

FIGURE 6b illustrates a preferred embodiment utilising DMX control of 
lighting device 62 which may comprise an individual lighting device or a number of 
devices connected via DMX. Accordingly, a number of DMX channels 51.2(a) are 
dedicated to each individual lighting device 62, and in the preferred embodiment, at 
least one DMX channel 51. 2(a) is available for each controllable aspect of an 
individual lighting device 62, for example, pan, tilt or strike. Within output arranger 
51.2, timing and data buses 25 and 10 are made available to be routed to through to 
DMX channels 51.2(a) as illustrated Connection node 51.2(b) illustrates a preferred 
embodiment of a routing form that enables any combination of coimections. Output 
arranger 51.2 makes this routed data available to function & parameter controls 51.1 
whereupon each data line, may be individually treated by a mmiber of functions and 
or parameter controls, such controls may have, but are not limited to, the following 
options; 

The first aspect of the aforementioned options is function, including; 
toggle, slow ramp, fast ramp, logical, counter, fader, sound to light, colour change, 
gobo change, do nothing, flash, home, blackout or user defined. 

The second aspect is parameter, including; on for 'n' off for *n\ over 'n' 
triggers; AND/NAND/OR and other logical functions; count 'n' triggers, on under 
threshold; on over threshold; trigger from 'n' while logically high and other logical 
combinations; and, go to colour 'x'. The third aspect is cycle, including; cycle, one- 
shot, forward/back, cycle 'n' times and other cycle connotations. All aspects and 
options are available in a form that enables countless combinations of options 
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including inverses and cross correlation between aspects. Data from moods 5 1 forms 
output bus 55. 

As illustrated in FIGURE 1, output bus 55 is fed to output devices 60 and 
on to lighting devices 62 via connections 61. 

5 Output devices 60 may be one or more internal or external serial data 

cards or devices whereby data may be in the form of DMX or other lighting control or 
data formats. Connections 61 can be electrically conductive wires, fibre optic cables, 
wireless links or any combination thereof Furthermore connections 2 may utilise a 
combination of commimications interfaces such as the Internet or modem 

10 connections. Lighting devices 62 may be a single electronically controlled lighting 
device or one or more networks of electronically controlled lighting devices, or any 
combination thereof. 



CLAIMS 



1 . A method for controlling electronic lighting devices that uses 
data resulting from analysis of audio information as a basis for controlling said 
electronic lighting devices in accordance with said audio information. 

2. A claim as claim 1 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in 
synchfonicity with said audio iiifofmatidn, 

3. A claim as claim 1 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in sympathy 
with said audio information. 

4. A claim as claim 1 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that can be based on 
identification of said audio information. 

5. A claim as claimed in claims 1, 2, 3 or 4 wherein said activity 
indicators can be used as a basis for controlling electronic lighting devices. 

6. A claim as claimed in claims 1, 2, 3 or 4 wherein said activity 
indicators can be used as a basis for creating control command sets for electronic 
lighting devices 

7. A claim as claimed in claims 1 , 2, 3, 4, 5 or 6 wherein said 
activity indicators can be used as a basis for scheduling said control command sets for 
electronic lighting devices. 

8. A digital controlling system for electronic lighting devices that 
uses data resulting from analysis of audio information as a basis for controlling said 
electronic lighting devices in accordance with said audio information, comprising; 

electrical links and audio input devices to receive said audio information, 
a computer and software arrangement, and 

output devices and electrical links to command said electronic lighting 
devices. 



9. A claim as claim 8 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in 
synchronicity with said audio information, 

10. A claim as claim 8 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in sympathy 
with said audio information. 

11. A claim as claim 8 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that can be based on 
identification of said audio infofmation 

12. A claim as claimed in claims 8, 9, 10 or 1 1 wherein said activity 
indicators can be used as a basis for controlling electronic lighting devices, 

13. A claim as claimed in claims 8, 9, 10 or 1 1 wherein said activity 
indicators can be used as a basis for creating control command sets for electronic 
lighting devices 

14. A claim as claimed in claims 8, 9, 10, 1 1, 12 or 13 wherein said 
activity indicators can be used as a basis for scheduling said control command sets for 
electronic lighting devices. 

15. A computer-readable medium containing a computer program 
that performs a method for controlling electronic lighting devices that uses data 
resulting from analysis of audio infofmation as a basis for controlling said electronic 
lighting devices in accordance with said audio information. 

16. A claim as claim 15 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in 
synchronicity with said audio information. 

17. A claim as claim 15 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that are in sympathy 
with said audio infofmation. 

18. A claim as claim 15 wherein said analysis of said audio 
information may be used to determine relevant activity indicators that can be based on 
identification of said audio information. 



19. A claim as claimed in claims 15, 16, 17 or 18 wherein said 
activity indicators can be used as a basis for controlling electronic lighting devices. 

20. A claim as claimed in claims 15, 16, 17 or 18 wherein said 
activity indicators can be used as a basis for creating control command sets for 

5 electronic lighting devices 

21. A claim as claimed in claims 15, 16, 17, 18, 19 or 20 wherein 
said activity indicators can be used as a basis for scheduling said control command 
sets for electronic lighting devices. 

22. A claim as in any preceding claim, wherein said system creates 
10 and maintains a lighting show in synchronicity and sympathy with said audio 

petfofmance without human intervention. 

23. A claim as in any preceding claim, wherein said system creates 
and maintains a lighting show in synchronicity and sympathy with said audio 
performance without a Hghting console. 

15 24. A digital controlling system for electronic lighting devices 

substantially as described herein with reference to Figures 1 - 6 of the accompanying 
drawing. 
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